test(ingestion): assert the fixed Excel behaviour, not the defect - #485
Merged
Merged
Conversation
main went red at c0fbb08: 7064 passed, 2 failed. #478 found that preserve_leading_zeros was a no-op for clean_excel and, following the repo's convention of not using xfail, pinned the defect as it behaved: assert list(fd.clean_excel(xlsx)["zip"]) == [2134, 501, 10001] # DEFECT #480 then fixed that defect. Each PR was green on its own branch, and each was rebased on a main that did not contain the other, so nothing caught the pair until both had landed. The two assertions were tripwires for the fix, and the fix duly tripped them. Both now assert the repaired behaviour, and each docstring keeps the history so the tests still explain why they exist. The round-trip case gains nothing artificial -- it simply asserts that the Excel hop no longer undoes a correct CSV clean, which is the property that was missing in the first place. The opt-out and explicit-dtype paths are asserted alongside, so the fix cannot drift into applying unconditionally. Full suite 7066 passed / 0 failed, coverage 94.21%; ruff clean repo-wide.
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
kevincostner17
added a commit
that referenced
this pull request
Sep 20, 2026
A numeric code column with one blank cell loads from CSV as float64, so
Series.astype("string") renders 10000266 as "10000266.0" and the trailing 0
reads as an extra digit. Every row of a perfectly valid column then fails a
digit-only pattern:
GS1-008 gpc_brick_code, int64 -> passed, 0 violations, trust 0.25
GS1-008 gpc_brick_code, float64 -> FAILED, 3 violations, trust 0.0625
Same codes, same rule, four-fold trust drop, decided by whether one cell
happened to be blank. Both regex rules in the repo were affected: GS1-008
([0-9]{8}) and FIN-008 ([A-Za-z0-9]{4,12}).
The repository had already settled the intended behaviour. retail/validator.py
carried _integral_float_text for exactly this case on the GTIN checks, and its
docstring describes this same CSV-blank-cell scenario -- the shared rule engine
simply never used it. The helper now lives in domains/base.py as
integral_float_text, _check_regex applies it, and retail imports it rather than
keeping a second copy.
Deliberately narrow: only an integral, finite float is rewritten. A genuine
decimal keeps its fraction, NaN and ±inf pass through (int(nan) would raise),
and text, integers and None are untouched -- so zero-padded strings keep their
padding and a real violation is still reported. Verified: a frame of genuinely
invalid codes still fails with all four rows flagged.
Three assertions in the new test file fail on main, across both affected rules.
Also updates test_domain_validation_lane.py, which pinned this defect as
current behaviour when #482 found it. That test is a tripwire for the fix and
the fix duly tripped it, so it now asserts the repaired behaviour with the
history kept in its docstring. This is the second time a pinned-defect test has
had to be flipped by the fix that resolved it (see #485); worth checking for
others before landing a behaviour change.
Full suite 7107 passed / 0 failed, coverage 94.94%; gauntlet gates all pass;
ruff clean repo-wide.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
main is red — this fixes it
mainatc0fbb08: 7064 passed, 2 failed.Why
#478 found that
preserve_leading_zeroswas a no-op forclean_exceland — following the repo's convention of not usingxfail— pinned the defect as it behaved:#480 then fixed that defect.
Each PR was green on its own branch, and each was rebased on a
mainthat did not contain the other, so nothing caught the pair until both had landed. Those two assertions were tripwires for the fix, and the fix duly tripped them.The change
Both assertions now describe the repaired behaviour, and each docstring keeps the history so the tests still explain why they exist.
The round-trip case gains nothing artificial — it simply asserts that the Excel hop no longer undoes a correct CSV clean, which is the property that was missing in the first place. The opt-out (
preserve_leading_zeros=False) and explicit-dtypepaths are asserted alongside, so the fix cannot drift into applying unconditionally.Verification
tests/test_ingestion_roundtrip.py: 22 passed.ruff check .clean repo-wide.Process note
A green CI check on each PR independently is not evidence that
mainwill be green after both land. When a test pins a defect as current behaviour, it is a merge-order dependency — the second of the pair needs a rebase onto the first and a re-run before merging.